DOCUMENT RESUME 



ED 284 388 



EC 200 205 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 

PUB DATE 

GRANT 

NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



Ysseldyke, James E.; And Others 

Policy Analysis of Screening and Referral for Early 
Childhood Special Education Programs. Research Report 
#11. Early Childhood Assessment Project. 
Minnesota Univ., Minneapolis. 

Special Education Programs (ED/OSERS), Washington, 
DC. 

Sep 86 
G008400652 

40p.; For other reports in this series, see EC 200 
204-209. 

Reports - Research/Technical (143) 
MF01/PC02 Plus Postage. 

Case Studies; Comparative Analysis; 'Disabilities; 
Early Childhood Education; Educational Diagnosis; 
'Handicap Identification; 'Referral; Regional 
Characteristics; 'School Policy; 'Screening Tests; 
'Special Education; Student Evaluation 

ABSTRACT 

Ecological case study methods were used in four early 
childhood special education (ECSE) screening programs to identify 
factors influencing outcomes of screening. Observations, interviews, 
file searches, and parent surveys were used to obtain 
multidimensional descriptions of ECSE programs, including outreach, 
screening, and referral practices. Two outcomes of screening were 
used as benchmarks for comparing programs: the percentage of eligible 
preschoolers screened (participation rate); and the percentage of 
screened children referred for further developmental assessment 
(referral rate). Results indicated that screening and referral 
practices varied considerably among programs, although few practices 
consistently influenced screening outcomes. Participation rates were 
clearly related to school district size and the accessibility of 
screening to the public. Referral rates were most obviously 
influenced by screening purposes, second-level referral decisions, 
separate speech/language referral criteria, and subjective judgment. 
Relationships among community agencies influenced both participation 
and referral rates. Considerations and guidelines are presented for 
evaluation and improvement of ECSE screening programs. (Author) 
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Abstract 



Ecological case study methods were used in four early childhood 
special education (ECSE) screening programs to identify factors 
influencing important outcomes of screening. Observations, 
interviews , file searches , and parent surveys were used to obtain 
multidimensional descriptions of ECSE programs, including outreach, 
screening, and referral practices. Two outcomes of screening were 
used as benchmarks for comparing programs: (a) the percentage of 
eligible preschoolers screened (participation rate), and (b) the 
percentage of screened chi 1 dren referred for further devel opmental 
assessment (referral rate). Results indicated that screening and 
referral practices varied consideraoly among programs, although few 
practices consistently influenced screening outcomes. Participation 
rates were most clearly related to school district size and the- 
accessibility of screening to the public. Referral rates were most 
obviously influenced by screening purposes, second-level referral 
decisions, separate speech/language referral criteria, and subjective 
judgment. Relationshi ps among community agencies influenced both 
participation and referral rates. Considerations and guidelines are 
presented for evaluation and improvement of ECSE screening programs. 
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Policy Analysis of Screening and Referral for 
Early Childhood Special Education Programs 

James E, Ysseldyke, Martha L. Thurlow, Patrick J. O'Sullivan, 
Jill A, Weiss, Paula A. Nania, and Camilla A, Lehr 

Since the Child Find Mandate of P.L. 94-142 directed states to 
locate, identify, and evaluate handicapped children in 1975, most 
states have implemented some form of preschool or early school 
screening program to facilitate the location and identification of 
handicapped preschoolers (Grac^y, Azzara, & Rheinherz, 1984). 
Preschool screening functions as an initial sorting procedure that 
identifies children from the general population who may have special 
education needs, or be "at-risk" for psychoeducationsl problems. 
Screening 1s composed of three procedural components or stages; (a) 
outreach procedures aimed at notifying and gaining the participation 
of the populace, (b) screening procedures and test administration 
aimed at identifying at-risk children, and (c) deciding whether and 
where to refer children for further evaluation. 

The increase in screening programs has been associated with 
increases in services for handicapped preschoolers. Forty-two states 
now mandate special education services to some portion of tne 
preschool handicapped population (U.S. Department of Education, 1985). 
And, the incentive for states to serve all handicapped children aged 3 
to 5 was strengthened by the Education of the Handicapped Amendments 
of 1986. In those amendments Congress established a new federal 
discretionary program to help states develop and implement a 
comprehensive, coordinated, interdiscipl inary program of early 
intervention services for handicapped infants, toddlers, and their 
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families. Hence, it seems likely that preschool screening will 
continue to be the primary gateway to a growing number of early 
childhood special education services. 

Dospite the widespread interest and efforts in screening 
oreschoolers, a variety of concerns about screening have been raised. 
These include the appropriateness of at-risk models (Keogh & Daley, 

1983) , the n*ed to clarify screening purposes (Gracey et al., 1984), 
and the accuracy of screening instruments (Lichtenstein & Ireton, 

1984) . In addition, studying the effects of screening services has 
been complicated by the seemingly haphazard grouping of federal and 
state services offered to handicapped and at-risk preschoolers and 
their families (Smitl , 1986). Perhaps these considerations and 
complications have contributed to the limited research addressing 
factors ohat may influence the outcomes of screening. In fact, 
determining now the many potential social, economic, and educational 
influences on screening actually affect screening outcomes has proven 
to be difficult (Ysseldyke & O'Sullivan, in press). For example, a 
study of screening programs with extremely high or low referral rates 
revealed no clear-cut variables related to the extreme variability in 
referral rates (Ysseldyke, Thurlow, Weiss, Lehr, & Bursaw, 1985). In 
different programs, different factors embedded in the context of 
unique community characteristics seemed to influence whether high or 
low percentages of screened children were referred for further 
evaluation. Thus, we are still faced with such fundamental questions 
as, "What actually happens during screening?" and "How does what 
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happens during screening influence screening outcomes?" Use of a 
research method capable of capturing relationships among many 
interacting variables appears necessary to answer these questions. 

In an effort to understand how many social, political, economic, 
and educational factors influence preschool screening, we studied 
several screening programs in depth, using an ecological case study 
approach. Quantitative and qualitative data were gathered in four 
screening programs from many sources using a variety of methods to 
obtain a rich multidimensional picture of each preschool screening 
program. Current practices used in each program during outreach, 
screening, and referral procedures were identified. Then, the 
practices were analyzed using two outcomes of screening as benchmarks 
for comparing programs: (a) the percentage of eligible preschoolers 
screened in each program (participation rate), and (b) the percentage 
of screened children referred for special education assessment 
(developmental referral rate). The degree of community participation 
in screening, and screening referral rates are clearly important 
factors that influence whether handicapped preschoolers ar, entified 
during screening. These benchmark variables were used to address 
three research questions: 

1. What factors influence participation in preschool screening 
programs? 

2. Do factors related to screening procedures (e.g., setting, 
personnel, tests) influence referral rates? 

3. Do screening referral criteria and practices influence 
screening referral rates? 

Findings related to these questions were integrated to produce a 
set of considerations and guidelines for policy makers and 
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practitioners. The considerations and guidelines also set a framework 
for evaluating the impact of specific factors on the identification of 
handicapped preschoolers in local school districts. 

Method 

Subjects 

Four school districts in Minnesota were subjects in this 
investigation. Within these districts, the focus was on the early 
childhood special education program (ECSE) and the preschool screening 
program. Many individuals within and outside these programs provided 
information for the study. The general characteristics of t'le four 
programs are deccribed here using fictitious community names to help 
readers follow the discussion. 

Oakwood is a rural community of approximately 6,000 residents. 
About 94% of the families in the district are above the poverty level 
(Bureau of the Census, 1982). The school district enrolls 
approximately 1,500 pupils in pre-kindergarten through grade 12 
programs each year, about 2% of whom are minorities. In the most 
recent academic year, 1985-86, there werr* 14 children enrolled in the 
Oakwood ECSE program. 

Elmwood is a suburban school district that serves primarily a 
middle to upper-middle class population, with almost 50% of the 
families earning an income above $30,000 (Bureau of the Census, 1982). 
The school district serves approximately 7,000 students in pre- 
kindergarten through grade 12 classes each year; only 2% of whom are 
minorities. Each year, the Elmwood ECSE program serves approximately 
30 children. 
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Birchwood Is a suburban school district that encompasses six 
communities. The total district population is approximately 33,763, 
with 99% of its families above the poverty level (Metropolitan 
Council, 1985). The school district enrolls approximately 8,400 
pupils in pre-kindergarten through grade 12 programs each year. In 
the 1985-86 academic year, 42 children were enrolled in the Birchwood 
ECSE program, which includes two components serving handicapped 
preschoolers. 

Maplewood is a large urban school district serving more than 
35,000 pupils from pre-kindergarten to grade 12. Compared to other 
districts in Minnesota, Maplewood falls at the 42nd percentile for 
median family income, and the 12th percentile in median age of 
residents (Bureau of the Census, 1982). About 38% of the residents 
belong to minority groups, and 45% of special education preschoolers 
are minorities (School District Information, 1985). 1 Within the early 
childhood special education program, School 1 serves an average of 
about 200 handicapped 4-year-old children in 13 classrooms during the 
school year. These children attend half-day sessions five days per 
week for up to nine months. About 90% of all handicapped preschoolers 
attend School 1, and the remaining severely handicapped preschoolers 
attend School 2 (physical handicaps), School 3 (hearing-impaired), and 
School 4 (autism). 

Preschool Screening (PSS) Program 

Except where otherwise noted, the term "screening" refers to the 
procedures associated with the PSS program. The four districts in 
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this study participated in the statewide PSS program administered by 
the state education department. The purpose of PSS, according to 
state statute, is to assist parents and communities with improving the 
health of children, and planning educational and health programs. 
Many districts also use PSS to meet the child find mandate of federal 
law requiring that handicapped preschoolers be identified. The PSS 
program is governed by state law and rules requiring all school 
districts to offer health and developmental screening to all children 
at least once before kindergarten entry. Districts are required to 
offer rix screening components: vision, hearing, growth 

(height/weight), health history, and developmental screening, and a 
summary interview with parents. Development is screened in four 
areas, including speech/language, cognitive, gross and fine motor, and 
social-emotional development. PSS is voluntary for parents and not 
required for school enrollment. Districts are reimbursed for 
screening costs, up to a max-' mum of $15.60 per child for fiscal year 
1985. 

One or more optional components, in addition to the six mandated 
screening components, were offered by three of the four programs 
studied. Oakwood offered a dental inspection to all children 
screened. Birchwood provided nutrition screening as part of the 
health history component. Maplewood offered four optional components 
to children eligible for medical assistance: dental, nutrition, 
physical inspection, and laboratory tests (urinalysis, blood tests). 



Procedure 

The four sites were selected to reflect a range in demographic 
characteristics (including community characteristics, and size of 
program) and in approaches to diagnostic assessment. Also, because 
data collection procedures required extensive contact with the sites, 
proximity to the research center was considered in this selection 
process. All sites contacted agreed to participate in the research. 

One research team member was assigned to each participating 
district and acted as primary contact person and data collector. 
Typically, more than one person collected data in each site. In three 
of the districts, data collection took place during an eight-month 
period. Due to time and travel restrictions, all data from the rural 
site (Oakwood) were collected during a three-month period in the 
spring. 

Data col 1 ection procedures incl uded : (a ) observations of 
meetings, classroom activities, and screening and assessment 
procedures, (b) extensive interviews with various staff and 
administrative personnel, (c) file searches, and (d) parent surveys. 
Although specific data collection procedures varied as a function of 
differences in the programs, the same research questions were asked in 
al 1 sites. Detailed information describing preschool screening, 
diagnostic assessment, the instructional programs, program exit 
procedures, and follow-up data on student participants was gathered 
for each site (see Ysseldyke, Thurlow, Lehr, Nania, O'Sullivan, Weiss, 
& Bursaw, 1986, for the full descriptive reports.) 



10 



8 

Results and Discussion 
What Factors Influence Participation ? 

Table 1 is a summary of the percentage of eligible children 
screened (participation rate), number of children screened, principal 
outreach methods used by programs, and the time of year and number of 
days screening was offered. 2 Participation rates were similar for 
Elmwood and Maplewood programs, slightly lower for Birchwood, and 
markedly higher for Oakwood. The relatively high participation rate 
in Oakwood was also above average rates observed statewide, which have 
varied between 78% to 84% over the past five yearo (Minnesota 
Department of Education [MDEJ , 1985). The high participation rate 
was, however, in general agreement with state education department 
observations that most districts, and particularly smaller-sized 
districts, screen over 90% of eligible preschoolers (MDE, 1985). Thus, 
district size, which is related to the number of children screened, 
appeared to be an important influence on participation rates. 

Although participation rates for the four programs varied 
considerably, outreach methods used to notify the public about 
screening were similar. Three of the programs relied on census data 
to obtain lists of residents. The residents were contacted by letter 
in all three districts. in two of these programs (Elmwood, 
Birchwood), eligible families also were contacted by telephone. All 
programs used a variety of additional outreach methods including 
notices in newspapers, newsletters, and flyers. However, outreach 
methods were not clearly related to participation rates. 
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Percentage of Eligible Children Screened (Participation Rate), 
Number of Children Screened, Principal Outreach Methods, 
Time- of Year, and Number of Days Screening Was Offered 



Programs 





Oakwood 


Elmwood 


Birchwood 


Maplewood 


rarLicipanon Rate 


88% 


66% 


54% 


61% a 


Number of Children Screened 


1 125 


358 




9 171 


Primary Outreach Methods 


Letter 

(Census) 

Newspaper 


Letter 
Telephone 
(Census) 
Newsletters 


Letter 
Telephone 
(Census) 
News paper 


Day Cares, 

Flyers, 

Media 


Time of Year 


April or May 


Sept. -Nov. 
March-May 


Feb, , March 


Sept. -May 


Number of Days Offered 


5 


50 


11 


160 



Estimated (see Footnote 2) 
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There was moderate evidence that participation rates were 
Influenced by screening schedules and accessibility of screening to 
the public. All programs screened children in the spring. Elmwood 
also offered screening 1n the fall, and Maplewood offered screening 
throughout most of the school year. Although Elmwood and Birchwood 
programs screened s1m1 1 ar numbers of ch1 1 dren , Elmwood offered 
screening on many more days, which may have contributed to the higher 
participation rate 1n Elmwood. Also, Maplewood was the only program 
to offer screening at multiple community sites. This use of multiple 
screening sites 1n Maplewood undoubtedly made screening more 
accessible for some Maplewood residents. In addition, by screening 1n 
Maplewood day cares, many- children of parents who worked or attended 
school were screened without parents taking time off from their other 
commitments. These factors probably helped to Increase the Maplewood 
participation rate. Unfortunately, 1t 1s not possible to determine 
whether participation rates were Increased by these factors, because 
Maplewood also used outreach procedures that differed somewhat from 
other communities. Nonetheless, a moderate degree of evidence 
suggested that factors related to screening accessibility contributed 
to variability 1n participation rates. 

To summarize, most screening programs used similar methods to 
notify the public about screening, although participation rates varied 
widely. For larger suburban and urban communities, participation 
rates ranged from about 50* to 65% of eligible children, regardless of 
the outreach method used. Although there are not establ 1shed 
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guidelines for participation rates, most screeners and screening 
coordinators said they wished that their participation rates were 
higher. There was some evidence that the accessibility of screening 
programs contributed to increased participation rates. Perhaps 
parents view a limited screening schedule as being inconvenient, too 
restrictive in relation to their schedules, or they may simply forget 
when screening is offered. Clearer evidence existed to indicate that 
school district size was a critical factor, one that limited the 
effectiveness of typical outreach efforts to smaller-sized 
communities. Thus, reliance on census data to notify the public led 
to a high participation rate only for the smallest screening program. 
The lower 'rates of family mobility and close community ties typically 
found in rural areas may have contributed to the higher participation 
rate in the rural program. 

Another possi bl e inf 1 uence on partici pation rates , rel ated to 
screening at multiple sites, involved relationships with community 
agencies. Thus, Maplewood relied primarily on contacts with day 
centers to reach families with eligible children, and obtained a 
participation rate comparable to other programs, which used census 
data to locate el igible chil dren. Apparently, collaboration with 
agencies already serving a portion of the preschool population can 
lead to increased participation in screening. In addition, the 
program with the lowest participation rate did not formally notify 
community agencies about screening. 
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Do Screening Procedures Influence Referral Rates? 

Time and setting . Factors related to the time and setting of 
screening, and referral rates are presented in Table 2. Developmental 
referral rates for Elmwood and Maplewood were consistent with the 
statewide average of 11X to 12% (MDE, 1985). However, referral rates 
for Oakwood and Birchwood programs were markedly below the state 
average. It should be noted that the Birchwood referral rate did not 
include referrals for assessment of speech/language concerns only, and 
consequently should be considered an underestimate of the overall 
referral rate for the program. 

Variation in screening sites was not related to referral rates in 
any obvious way. Oakwood, Elmwood, and Birchwood programs conducted 
screening in the school gym or cafeteria. In Birchwood, vision and 
hearing screening were conducted in adjacent band rooms, and children 
were registered in the main hallway outside the gym. In Maplewood, 
screening was conducted at about 80 different sites, including 40 city 
recreation centers, and about 40 day care settings. The facilities 
and quality of the atmosphere varied considerably among the various 
settings in Maplewood. Sometimes physical inspections were 
complicated by frigid temperatures or hearing screening by a noisy 
efwironment. Moreover, in all programs the screening environment 
sometimes presented di stractions that interfered wi th screeni ng 
procedures, especially during busier time periods. 

The average time used to screen each child varied between 30 
minutes for the Elmwood and Maplewood programs, to 120 minutes at 
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Table 2 

Description of Screening Setting, Average Time to 
Complete Screening, and Participation and Referral Rates 



Programs 



Time/Place Factors 



Oakwood 



Elmwood 



Birchwood 



Mapl ewood 



Developmental Referral Rate 1% - 3% 



11% 



5% 



11% 



Where Conducted 



Gym 



Ca feteria 



Gym 

Band Rooms 
Hallway 



Varies 



Average Screening Time 



90 minutes 30 minutes 120 minutes 



30 minutes 
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Birchwood. Much of the variance in screening time was due to 
differences in time spent talking with parents in the summary 
interviews, and to a lesser extent in time for parent completion of 
forms. For example, parents in Birchwood were asked to complete a 
50-item communication abi 1 i ties questionnaire, and to provide 
nutrition information on the health history form, which increased 
screening time. Developmental referral rates tended to be higher in 
programs with shorter average screening times. But there is little 
logical or empirical basis for this inverse relationship between 
average screening times and referral rates. The apparent 
relationship, in fact, may be coincidental. Influences on referral 
rates that are more obvious and compelling are presented below. 

Personnel . Table 3 is a summary of developmental referral rates, 
screening personnel, personnel who conducted summary interviews, and 
referral decision makers for the four programs. Two of the programs 
(Elmwood, Birchwood) relied heavily on volunteers to carry out 
screening procedures, and nurses to summarize results to parents. In 
these programs, children were referred to multidisciplinary teams. In 
Maplewood, two nurses conducted all screening components, including 
the summary interview with parents. In Oakwood, six different types 
of professionals conducted screening, and the Early Childhood (EC) 
Coordinator summarized results and made referrals. The developmental 
component at Oakwood was administered in sections by the EC teacher, a 
nurse, and speech/language pathologist. A dental hygenist carried out 
the dental component, and technicians were contracted through the 
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Table 3 

Screening Personnel, Decision Makers, Participation and Developmental Referral Rates 



Programs 





Oakwood 


Elmwood 


Birchwood 


Maplewood 


Developmental 
Referral Rate 


1% - 3% 


11% 


5% 


11% 


Screening Personnel 


EC Teacher 

Speech Clinician 

Nurse, Dental 

Hygenist, 

2 County Techs. 


15-20 Volunteers 


15 Volunteers 
Several Nurses 


2 Nurses 


Summary Jntervit ^s 


EC Coordinator 


2 Nurses 


2 Nurses 


2 Nurses 


Referral Decision 
Makers 


EC Coordinator 


Mul tidisci- 
plinary Team 


Mul tidisci- 
plinary Team 


2 Nurses 
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County to perform hearing and vision screening. Interestingly, this 
program was the smallest screening program, but employed the highest 
number of di f f erent personnel . The 1 argest program (Map! ewood ) 
employed the fewest different personnel . There was no apparent 
relationship between the types or numbers of personnel involved in 
screening and developmental referral rates. 

S creening tests and procedures . Tab! e 4 i s a summary of 
information about screening tests and procedures in relation to 
referral rates. Elmwood and Map! ewood used a single screener to 
administer all screening components to each child. Oakwood and 
Birchwood used a station administration approach children moved 
from one station to the next with each screening component being 
administered by different screening personnel. These differences in 
administration correspond to the recommended administration procedures 
associated with the developmental screening tools. These latter two 
programs used the DIAL (Mardell & Goldenberg, 1975) or its revised 
version, DIAL-R (Mardell -Czudnowski & Goldenberg, 1983), which were 
standardized using a station approach. In contrast, the DDST 
(Frankenburg & Dodds, 1967) used in Elmwood and the loca^ screening 
tool (LST) used in Maplewood were standardized by having one screener 
administer the entire test. The DIAL, DIAL-R, and DDST are widely 
used screening instruments. Only the LST used in Maplewood was 
standardized and normed on the local population. The technical 
adequacy of popular screening instruments has been addressed 
previously (Lehr, Ysseldyke, & Thurlow, 1986; Thurlow, O'Sullivan, & 
Ysseldyke, 1986). 
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Table 4 

Screening Tools, Procedures, and Developmental Referral Rates 
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Programs 





Oak wood 


Elmwood 


Ri rchwood 




Developmental Referral 


1% - 3% 


11% 


5% 




Administration Procedure 


Station 


1 Screener/Child 


Station 


1 Screener/ChUd 


Developmental Tool 


DIAL b 


DDST a 


DIAL-R b 


Local Tool (LST) 


Rescreening 


Re-DIAL 
(except speech) 


Vision , 
Hearing 


Vision, 
Hearing 


Hearing 



a 00ST: Denver Developmental Screening Test 

b DIAL and DIAL-R: Developmental Indicators for the Assessment of Learning, and revised 
version 
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Oakwood and Birchwood programs, which used the station approach 
of the DIAL or DIAL-R, had lower referral rates than the other 
programs. However, it appears likely that this relationship between 
screening instruments and referral rates was due to differences in the 
referral process of programs (see below). Referral criteria for 
popular screening tests have been selected by test authors to yield 
roughly comparable referral rates. In fact, similar referral rates 
for diffeent popular screening tests have been reported in the 
literature (e.g., Lichtenstein, 1981). 

Rescreening practices reduced referral rates. Three of the 
programs routinely rescreened children who failed hearing screening, 
and two of them also rescreened vision failures. The other program 
(Oakwood) rescreened children who failed the Conceptual and Motor 
areas of the DIAL. Children who failed the Language area of the DIAL 
were referred for in-depth speech/language assessment at Oakwood. It 
is noteworthy that the only program that rescreened in the 
developmental area had the lowest developmental referral rate. 

In summary, screening time, setting, personnel, procedures and 
instruments varied widely among the four programs. Average screening 
times ranged from 30 to 120 minutes depending on the length of summary 
interviews, and the length and number of forms parents completed. In 
all programs screening was sometimes compl icated by setting 
conditions, such as a no sy environment. Some programs used 
volunteers to conduct screening, other programs employed 
professionals. From one to six screeners administered screening 
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procedures, and each program used a different screening instrument. 
Only one program rescreened children in the developmental area, 
although rescreening in vision and hearing areas was more common. 
Percentages of eligible children referred for further evaluation did 
not appear to be affected by the screening setting or personnel used 
by different programs. The length of screening procedures and 
screening tests appeared to be related to referral rates. However, 
this relationship seemed to be a spurious one, not supported by 
expectations based on logic or previous research. In addition, more 
obvious influences upon referral rates are presented below. 
Rescreening of children who failed the developmental screening 
component was one factor that appeared to influence developmental 
referral rates — the referral rate was lower when two consecutive 
screening failures were required for a referral. 

Do Referral Critoria Influence Screening Outcomes? 

Developmental referral rates, developmental referral agents, and 
speech referral criteria and agents for each program are shown in 
Table 5. All programs used age-normed cut-off scores from screening 
tests as developmental referral criteria. However, Maplewood used 
criteria based on total developmental scores, and Oakwood and Elmwood 
programs relied on subtest or subarea cut-off scores. The Birchwood 
criterion emphasized the DIAL-R total score, although subarea scores 
also were considered in a subjective manner. Thus, even when 
screening tools were highly similar (i.e., DIAL and DIAL-R), referral 
criteria were not the same. Intuitively, one might expect referral 



Table 5 

Developmental and Speech/Language Referral Criteria, Referral Agents, and Referral Rates 



Programs 



Oakwood Elmwood Birchwood Maplewood 



Developmental Referral 
Rate 


1% - 3* 


11% 


5% 


11% 


Developmental Referral 
Criteria 


Fail All 3 
DIAL Areas 
(excluding 
Communication) 


Fail 2/3 
DDST Areas 


Fail or 
Within 5 
points of 
DIAL-R 


Below LST 
Total Score 
Cut-Off 


Speech/Language 
Referral Criterion 


Fail DIAL 

Communication 

Area 


Meet Develop. 
Criterion + 
Fail Speech 
Worksheet 


Fail DIAL-R 
Language Area 


Below LST 
Cut-Off + 
Screener 
Judgment 


Developmental Referral 
Agent 


Program 
Assessment 


Multidisci- 
plinary Team 


Mul ti disci- 
plinary Team 


Program 
Assessment 


Speech/Language 
Referral Agent 


Speech/ 

Language 

Assessment 


Program 
Assessment 
ur Screening 


Speech/Language 
Assessment 


Program 
Assessment 
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rates to be related to the leniency (or stringency) of referral 
criteria. However, comparison of referral criteria was complicated by 
the use of different scores or a combination of scores by each program 
in formulating their criterion. Despite this variability in criteria, 
referral rates were not related in any obvious way to whether programs 
used total scores versus subarea scores as referral criteria. 

All programs used separate referral criteria for speech/language 
assessment. In some programs speech/language referral criteria were 
applied in addition to developmental criteria, and in other programs 
speech/language criteria were applied separately from developmental 
criteria. In Maplewood, speech/language referral criteria were the 
same as developmental criteria (i.e., below the LST cut-off), with the 
additional criterion of low ratings in speech as judged by screening 
nurses. All children referred for speech/language assessment in 
Maplewood also participated in the ECSE classroom assessment. This 
was not the case at other programs. The Oakwood and Birchwood 
programs used failure on the Language subarea (OIAL-R) or 
Communication subarea (DIAL) as referral criteria for speech/language 
assessment. Children failing these subareas typically were assessed 
by the speech/language clinician. These children were not considered 
for the ECSE program, unless they also met the developmental referral 
criterion. In Elmwood, the speech/language referral criterion was the 
same as the developmental referral criterion (i.e., failure on 2/3 
DDST areas), but also included failure on the speech worksheet 
administered during screening. Speech/language referrals in Elmwood 
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were handled in a manner similar to developmental referrals, where a 
multidisciplinary team decided whether speech/language screening or 
assessment was appropriate. 

Thus, for two programs, speech/language referral criteria were 
relatively stringent compared to developmental criteria — referrals 
in the speech/language area required failure using a speech/language 
and developmental criterion. For the remaining programs, 
speech/language criteria were independent of developmental criteria 
and relatively lenient. Hence, a speech/language referral required 
failure in one screening test subarea, whereas developmental referrals 
required failure in multiple subareas or on a composite test score. 
Unfortunately, referral rates in the speech/language area were not 
available for analyzing the effects of separate speech/language 
criteria. However, it appears likely that use of separate and 
relatively lenient or strict referral criteria in the speech/language 
area should lead to differences between speech/language referral rates 
and referral rates in other developmental areas. In fact, statewide 
referral rates in the speech/language area consistently have exceeded 
referral rates in other developmental areas (MDE, 1985). 

Do Referral Practices Influence Screening Outcomes? 

Developmental referral agents and practices varied considerably. 
In Oakwood and Maplewood the referral agent was the Early Childhood 
Special Education (ECSE) assessment classroom. In Elmwood the 
referral agent was a multidisciplinary team, the members of which 
decided whether children should be briefly assessed or assessed in 
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depth by the ECSE team based on DDST subarea scores, information 
gathered in a home visit by the social worker, and any other available 
information. Elmwood children who met developmental referral criteria 
were assessed in the ECSE diagnostic classroom, or briefly "screened" 
in some developmental areas, 3 or both screened and assessed in 
different areas, or both screened and assessed in the same area (if 
the screener recommended further assessment). Birchwood also used a 
multidisciplinary team to decide what to do with children who failed 
PSS screening, or scored within five points of the failure criterion. 
In Birchwood, the PSS coordinator selected files to present at 
multidisciplinary team meetings in March, based on DIAL-R total and 
subarea scores, parent concerns, and observations by screeners. The 
Birchwood multidisciplinary team decided whether to refer children for 
a full ECSE classroom assessment, speech/language assessment alone, or 
no assessment. In 1986, 47 out of 597 screened children (8%) were 
selected by the Birchwood screening coordinator for discussion at the 
team meetings. The team recommended in-depth diagnostic assessment 
for 27 of the 47 children (5* of all children screened). PSS 
screeners and parents did not attend these initial team meetings in 
Elmwood and Birchwood. 

Lower referral rates were associated with programs using a 
second-level referral criterion, such as the judgment of a 
multidisciplinary team (e.g., Birchwood) or failure on a rescreening 
(Oakwood). It is not surprising that introduction of a second 
criterion, or second "hurdle" could lead to lower referral rates. 
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Elmwood also used judgment of a mult1d1$dpl1nary team, which reduced 
the number of children who were assessed in-depth by the ECSE team. 
However, all children referred to the Elmwood team were at least 
assessed briefly (I.e., "screened") 1n some developmental area. 

What Other Factors Influenced Referral Rates? 

In all programs, subjective data based on screener observations 
and parent reports also were considered 1n making referral decisions. 
In general, 1t was difficult to specify to what extent this subjective 
Information and cl Inical judgment Influenced screening referral 
decisions. However, the child's age, and parental and screener 
judgment were critical factors influencing referral decisions for 
questionable cases 1n all programs. In Maplewood for example, 
children close to kindergarten age were not referred for ECSE 
assessment, because they were too old for the Maplewood ECSE program. 
Instead, parents were told by screeners to inform the child's school 
for the next year about screening failures. In the other programs, 
younger children were more likely not to be referred f * further 
assessment. Thus, in Oakwood, Elmwood, and Blrchwood, the decision to 
assess children around 3^-years-old was more likely to be postponed, 
especially 1f the parent or referral decision maker felt the child 
might need more time to mature. In such cases, younger children could 
be rescreened the following year. 

The purpose of screening, as it was Implicitly applied in each 
program, appeared to be another important Influence on referral rates. 
Recall that the purpose of the statewide PSS program was defined 



27 



25 

broadly, and did not specifically emphasize locating and Identifying 
handicapped children. However, most programs relied on the PSS 
program for Identifying most children who were later more fully 
assessed and Identified as handicapped. The Oakwood program was an 
exception. in Oakwood, most handicapped children had been referred 
from a local Developmental Achievement Center (DAC) where they had 
been Identified and served at an earlier age. As a result, these 
children in the DAC did not participate in the PSS program, and 
instead were directly referred fe.- ECSE program assessment by DAC 
personnel. Thus, DAC eligibility procedures were the principal method 
of identifying potentially handicapped children in Oakwood. And the 
Oakwood screening referral rate was reduced markedly, because few 
children considered for the ECSE program participated in preschool 
screening. 

In summary, four factors appeared to account for most of the 
variation in referral rates. One involved the purpose of screening, 
as evidenced by the degree to which programs relied on screening 
referrals for identifying potentially handicapped children. In the 
program with the lowest referral rate, children referred from DACs, 
not screening, were admitted most frequently to the ECSE program. In 
this program children in DACs were identified as being at-risk at an 
earlier age, and often continued to receive services in the ECSE 
program after terminating the DAC program. Apparently, these children 
were preselected or "pre-screened ," and did not participate in the PSS 
program. This example points out again the influence that 
relationships with community agencies can have on screening outcomes. 
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Another factor contributing to variation in referral rates was 
whether programs used some kind of second-level screening or decision 
process to reduce the number of children referred for assessment. 
Second-level decision making took a number of forms. In one program 
children were rescreened in the developmental area, and in several 
programs children who failed screening were referred to 
multidisciplinary teams for further deliberation. In each of these 
cases, the number of children participating in the program's in-depth 
assessment procedures was reduced by second-level screening or 
decision-making practices, which introduces an intermediate 
decision/assessment stage between "screening" and "assessment." The 
efficacy of second-level screening has some support in the literature 
(e.g., Teska & Stoneburner, 1980), although the appropriateness of 
specific procedures used in specific programs needs to be evaluated. 

Third, use of separate referral criteria and procedures in the 
speech/language area may have influenced the number of children 
referred for further assessment. Although referral rate data in the 
speech/language area were not available for all programs, the 
available data and impressions of school personnel support the notion 
that separate speech/language criteria are sometimes relatively 
lenient, or more dependent on clinician judgment. Consequently, a 
higher proportion of children may be referred because of 
speech/language concerns compared to other developmental areas, which 
is a finding observed in statewide PSS referral rates (MDE, 1985). 

Finally, subjective judgments of screeners, referral decision 
makers, and parents influenced screening referrals, particularly in 
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questionable pass/fail cases, and as a function of the child's age. 
Although all programs screened children between 3%- and 5-years-old, 
different programs tended to focus on detecting potential problems of 
children at either end of the age range. Most programs were less 
concerned about younger children in the "borderline" pass/fail region, 
reasoning that the children may need more time to mature and could be 
rescreened the following year. However, one program tended to be less 
concerned in similar cases with older children, because the ECSE 
program excluded older children. These older children could be 
assessed upon kindergarten entry. But in all programs and across all 
ages screened, the subjective impressions of parents and judgments of 
screeners were more likely to influence referral decisions in 
borderline pass/fail cases. In general, the influence of subjective 
information varied among programs and among those involved in each 
program. 

Considerations and Guidelines 
Factors identified in this study as important influences on 
screening outcomes deserve particular attention because, for the most 
part, they have been ignored in the literature. And, factors that 
influence screening outcomes require more. thoughtful consideration by 
researchers and practitioners, because screening has become widespread 
and affects so many children. Furthermore, screening is typically the 
first contact between families and the educational system, and the 
only contact where the general populace is scrutinized for possible 
special education assessment. Thus, screening outcomes and their 
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influences have broad-based implications that need to be the focus of 
more intensive analysis. Findings of the present study using in-depth 
case study methods suggest that similar approaches may be applied 
successfully in other school districts. 

Screening and referral practices varied considerably among 
programs, although few practices appeared to have consistent effects 
on screening outcomes. In each program the different practices seemed 
justifiable in the local context, and the impact of practices was 
dependent on the local context. Across all programs, screening 
participation rates were most obviously influenced by school district 
size, and the accessibility of screening to the public. Screening 
referral rates appeared to be influenced by the purpose of screening, 
use of second-level decision making, separate speech/language referral 
criteria, and subjective judgment. Both participation and referral 
rates were influenced by relationships with community agencies. These 
findings are encouraging given that previous research has had 
difficulty identifying influences on screening outcomes (e.g. , 
Ysseldyke & O'Sullivan, in press). 

Even though screening and referral practice variability was 
remarkable, there are reasons to believe that practices in other 
states may be more variable. Consider that the school districts 
studied participated in a longstanding state-administered screening 
program that specified the general purposes of screening, age range of 
children screened, the type and number of health and developmental 
areas to be screened, and specific screening procedures in some 
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screening areas (particularly heal th-related areas). Also, annual 
training workshops for screeners were conducted by the state health 
department. It seems likely, then, that screening and referral 
practices could be more variable in other states, and that the 
increased variability would be associated with even more potential 
influences on the outcomes of screening. Thus, factors that did not 
appear to influence screening outcomes in this study (e.g., outreach 
methods, screening personnel, screening tests), and factors not 
germane to this study (e.g., screening domains) may indeed influence 
the impact of screening in other locales. Such factors should not be 
considered as inconsequential contributors to the screening outcomes 
in general . 

The following guidelines are presented to researchers and 
practitioners interested in evaluating and improving screening 
programs. These guidelines are not meant to be exhaustive. Rather, 
they highlight some of the considerations and initial steps needed to 
provide data that can lead to more appropriate policy decisions 
regarding preschool screening. Alternative approaches to evaluating 
ECSE programs have been discussed by Sheehan and Keogh (1982). 

1. Define intended screening procedures, purposes, and goals . 

Screening procedures, purposes, and goals need to be specified 
for outreach practices, screening practices, and referral practices. 
These definitional issues can be discussed within screening programs, 
and with representatives from other agencies providing service to 
preschoolers. The general intent of discussion among agencies is to 
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reduce gaps and duplications in service and facilitate efficient, 
comprehensive service delivery. For example, in many communities the 
state PSS program is not suited to be the sole means of identifying 
potentially handicapped children, because many children are never 
screened by PSS programs. Consequently, districts can try to maximize 
participation in PSS, perhaps by experimenting with different ways to 
make screeni ng more accessi bl e to the publ ic , but al so can take 
advantage of other screening and referral networks in the community. 

Goals of screening should take into account the limited ability 
of screening to accurately identify needful children. Because of 
limits on the percentage of correct screening referral decisions 
(Gallagher & Bradley, 1972; Liechtenstein & Ireton, 1984), additional 
steps are necessary to "right" the wrong decisions. Screening 
children on more than one occasion using data from multiple sources 
has been recommended as one way to increase screening accuracy (Gracey 
et al., 1984). Also, important considerations about the relation 
between referral criteria and correct and incorrect screening 
decisions have been provided by Lichtenstein and Ireton (1984). The 
important thing to remember is that errors are introduced primarily by 
the limited measurement accuracy of screening tools, the inappropriate 
use of screening data and application of decision criteria, and the 
rapid developmental changes that young children undergo. These 
sources of error have to be taken into account to attain appropriate 
screening goals. 
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2- Describe and assess actual practices and ou tcomes as fully as 
possible . £ 

Determining who does what to whom with what kinds of outcomes is 
an important step in describing the existing context of service 
delivery, so that evaluation and possible modification of practices 
can occur. It may be helpful to draw flow charts depicting the 
relationships among service providers and the paths children follow 
through screening, diagnostic assessment, treatment, and program exit 
procedures. In addition to providing a program description, such 
efforts should help identify gaps in knowledge and needs for data 
collection. The characteristics and numbers of children eligible for 
screening, screened, not screene„, referred for assessment, not 
referred for assessment, assessed, not assessed, and so on, are 
critical outcome variables needed to evaluate the appropriateness and 
accuracy of screening programs. 

Data collection also can focus on influences upon screening 
participation and referral rates identified above. And where various 
agencies use different screening and referral practices, careful 
attention should be paid to definitional issues, such as what is 
"screening" or a "screening referral?" Qualitative data, such as the 
comments and reactions of parents, should also be collected to 
encourage consideration of a broad range of potentially relevant 
factors. Practices should be described fully enough so that their 
consistency can be assessed across programs, screeners, and children. 
And because the fielc* is changing rapidly, data need to be collected 
continuously over relatively long time periods. Although this 
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suggested record keeping may seem like extra work, much of the 
information is already gathered in many districts. What is needed is 
a more systematic and internally consistent means of identifying, 
organizing, and sharing important information for guiding policy 
decisions. 

3 - Evaluate actual screening practices and outcomes in relat ion to 
intended procedures, purposes, and goTTs^ 

Often, screening outcomes can be readily compared to objectively- 
defined screening goals and purposes. When screening purposes and 
goals are ill-defined or weakly related to each other, problems with 
interpreting screening outcomes data will be apparent. Thus, the ease 
of interpreting and evaluating screening outcomes is often a yardstick 
for how successful the preceding definitional and descriptive steps 
have been. 

More often than not, outcome data will suggest new questions that 
need addressing, and data to be collected. Rather than being an 
endpoint, this evaluation phase should lead to recycling the 
evaluation process back to consideration of policy goals and purposes. 
Results may suggest that changes in screening practices are needed to 
meet intended goals. At other times, results may suggest that policy 
changes in purposes and goals are needed. In this case, informing 
local, state, and federal policy makers about obstacles to desired 
program objectives may be appropriate. 

In conclusion, many issues confront the complex array of service 
providers trying to meet the needs of children. Because federal, 
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state, county, and local agencies provide services to young children 
and their families, considerable effort directed toward planning and 
evaluating coordinated, comprehensive interagency service delivery 
will be necessary. Smith (1986) outlined three workable approaches to 
planning coordinated interagency services among programs serving young 
children. The guidelines presented above may help those interested 1n 
evaluating and Improving screening services for young children. If 
similar evaluation techniques can be applied to diagnostic assessment, 
treatment, and program exit procedures, policy makers could 
incorporate a more solid informational basis for policy decisions. A 
foundation of knowledge, Including data that describe outcomes for 
programs and children, appears necessary to guide the many challenging 
decisions awaiting this rapidly changing field. 
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Footnotes 

School district information sources included special education 

program handbook (1984, a school district annual report [1985] and 

student statistical report [19853, and a brochure £19853 about 

learning opportunities for preschoolers in the school district. 
2 

It should be noted that the number of children screened and the 
participation rate provided in Table 1 were derived in different ways. 
In Oakwood, the participation rate was estimated by the district's 
Early Childhood Coordinator. The participation rate for Maplewood was 
estimated by taking the number of children screened and dividing by 
the kindergarten enrollment for the same year. In other programs, 
participation rates were calculated by dividing the number of children 
screened by the number of children eligible for screening (based on 

census data). 
3 

In the Elmwood program, the term "screening" was also used to 
denote brief (e.g., one hour) assessment that typically followed PSS 
screening. For example, a child referred for ECSE assessment may be 
briefly screened in the speech/language area rather than thoroughly 
assessed, if no serious problems or concerns in speech/language skills 
were noted during PSS screening. In this context, screening was 
defined as "attending to" or observing a developmental area without 
using standardized assessment tools. 
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